Chromosome analysis is essential for diagnosing genetic disorders. For hematologic malignancies, identification of somatic clonal aberrations by karyotype analysis remains the standard of care. However, karyotyping is costly and time-consuming because of the largely manual process and the expertise required in identifying and annotating aberrations. Efforts to automate karyotype analysis to date fell short in aberration detection. Using a training set of ~10k patient specimens and ~50k karyograms from over 5 years from the Fred Hutchinson Cancer Center, we created a labeled set of images representing individual chromosomes. These individual chromosomes were used to train and assess deep learning models for classifying the 24 human chromosomes and identifying chromosomal aberrations. The top-accuracy models utilized the recently introduced Topological Vision Transformers (TopViTs) with 2-level-block-Toeplitz masking, to incorporate structural inductive bias. TopViT outperformed CNN (Inception) models with >99.3% accuracy for chromosome identification, and exhibited accuracies >99% for aberration detection in most aberrations. Notably, we were able to show high-quality performance even in "few shot" learning scenarios. Incorporating the definition of clonality substantially improved both precision and recall (sensitivity). When applied to "zero shot" scenarios, the model captured aberrations without training, with perfect precision at >50% recall. Together these results show that modern deep learning models can approach expert-level performance for chromosome aberration detection. To our knowledge, this is the first study demonstrating the downstream effectiveness of TopViTs. These results open up exciting opportunities for not only expediting patient results but providing a scalable technology for early screening of low-abundance chromosomal lesions.

意见摘要是创建摘要的任务，以获取用户评论中的流行意见。在本文中，我们介绍了Geodesic Summarizer（GeoSumm），这是一种新型系统，可执行无监督的提取意见摘要。 GeoSumm涉及基于编码器的表示模型，该模型将文本表示为潜在语义单元的分布。 GeoSumm通过在多个解码器层上对预训练的文本表示进行字典学习来生成这些表示。然后，我们使用这些表示形式使用新型的基于测量距离的评分机制来量化审查句子的相关性。我们使用相关得分来确定流行意见，以构成一般和特定方面的摘要。我们提出的模型GeoSumm在三个意见摘要数据集上实现了最先进的性能。我们执行其他实验来分析模型的功能，并展示跨不同域{\ x}的概括能力。

在这项工作中，我们介绍了内核化变压器，这是一个通用，可扩展的，数据驱动的框架，用于学习变压器中的内核功能。我们的框架将变压器内核作为光谱特征图之间的点产物近似，并通过学习光谱分布来学习内核。这不仅有助于学习通用的内核端到端，而且还可以减少变压器从二次到线性的时间和空间复杂性。我们表明，在准确性和计算效率方面，内核化的变压器实现了与现有的有效变压器体系结构相当的性能。我们的研究还表明，内核的选择对性能有重大影响，而内核学习变体是固定内核变压器的竞争替代方案，无论是长时间的序列任务。

Transformers-based models, such as BERT, have been one of the most successful deep learning models for NLP. Unfortunately, one of their core limitations is the quadratic dependency (mainly in terms of memory) on the sequence length due to their full attention mechanism. To remedy this, we propose, BIGBIRD, a sparse attention mechanism that reduces this quadratic dependency to linear. We show that BIGBIRD is a universal approximator of sequence functions and is Turing complete, thereby preserving these properties of the quadratic, full attention model. Along the way, our theoretical analysis reveals some of the benefits of having O(1) global tokens (such as CLS), that attend to the entire sequence as part of the sparse attention mechanism. The proposed sparse attention can handle sequences of length up to 8x of what was previously possible using similar hardware. As a consequence of the capability to handle longer context, BIGBIRD drastically improves performance on various NLP tasks such as question answering and summarization. We also propose novel applications to genomics data.

Reduced system dependability and higher maintenance costs may be the consequence of poor electric power quality, which can disturb normal equipment performance, speed up aging, and even cause outright failures. This study implements and tests a prototype of an Online Sequential Extreme Learning Machine (OS-ELM) classifier based on wavelets for detecting power quality problems under transient conditions. In order to create the classifier, the OSELM-network model and the discrete wavelet transform (DWT) method are combined. First, discrete wavelet transform (DWT) multi-resolution analysis (MRA) was used to extract characteristics of the distorted signal at various resolutions. The OSELM then sorts the retrieved data by transient duration and energy features to determine the kind of disturbance. The suggested approach requires less memory space and processing time since it can minimize a large quantity of the distorted signal's characteristics without changing the signal's original quality. Several types of transient events were used to demonstrate the classifier's ability to detect and categorize various types of power disturbances, including sags, swells, momentary interruptions, oscillatory transients, harmonics, notches, spikes, flickers, sag swell, sag mi, sag harm, swell trans, sag spike, and swell spike.

Human activity recognition (HAR) using IMU sensors, namely accelerometer and gyroscope, has several applications in smart homes, healthcare and human-machine interface systems. In practice, the IMU-based HAR system is expected to encounter variations in measurement due to sensor degradation, alien environment or sensor noise and will be subjected to unknown activities. In view of practical deployment of the solution, analysis of statistical confidence over the activity class score are important metrics. In this paper, we therefore propose XAI-BayesHAR, an integrated Bayesian framework, that improves the overall activity classification accuracy of IMU-based HAR solutions by recursively tracking the feature embedding vector and its associated uncertainty via Kalman filter. Additionally, XAI-BayesHAR acts as an out of data distribution (OOD) detector using the predictive uncertainty which help to evaluate and detect alien input data distribution. Furthermore, Shapley value-based performance of the proposed framework is also evaluated to understand the importance of the feature embedding vector and accordingly used for model compression

FP8是加速深度学习训练推论以外的16位格式的自然发展。在本文中，我们提出了一个8位浮点（FP8）二进制互换格式，该格式由两个编码组成-E4M3（4位指数和3位Mantissa）和E5M2（5位指数和2位指数和2位Mantissa）。尽管E5M2遵循IEEE 754惯例代表特殊值的惯例，但E4M3的动态范围是通过不代表无限态，只有一个Mantissa Bit-Pattern来扩展NAN。我们证明了FP8格式对各种图像和语言任务的功效，从而有效地匹配了16位培训课程所达到的质量。我们的研究涵盖了主要的现代神经网络体系结构 - CNN，RNN和基于变压器的模型，使所有超参数与16位基线训练课程保持不变。我们的培训实验包括大型，最多175b参数，语言模型。我们还检查了使用16位格式训练的语言模型的FP8训练后定量化，该格式抗拒固定点INT8量化。

在基于视觉的辅助技术中，具有不同新兴主题的用例，例如增强现实，虚拟现实和人类计算机互动等不同的主题中的用例中，自动眼目光估计是一个重要问题。在过去的几年中，由于它克服了大规模注释的数据的要求，因此人们对无监督和自我监督的学习范式的兴趣越来越大。在本文中，我们提出了Raze，Raze是一个带有自我监督的注视表示框架的区域，该框架从非宣传的面部图像数据中发挥作用。 Raze通过辅助监督（即伪凝视区域分类）学习目光的表示，其中目的是通过利用瞳孔中心的相对位置将视野分类为不同的凝视区域（即左，右和中心）。因此，我们会自动注释154K Web爬行图像的伪凝视区标签，并通过“ IZE-NET”框架学习特征表示。 “ IZE-NET”是基于胶囊层的CNN体​​系结构，可以有效地捕获丰富的眼睛表示。在四个基准数据集上评估了特征表示的判别性能：洞穴，桌面，MPII和RT-GENE。此外，我们评估了所提出的网络在其他两个下游任务（即驱动器凝视估计和视觉注意估计）上的普遍性，这证明了学习的眼睛注视表示的有效性。

现有的数据依赖性哈希方法使用具有数百万个参数的大型骨干网络，并且计算复杂。现有的知识蒸馏方法使用深（教师）模型的逻辑和其他功能，并将其作为紧凑型（学生）模型的知识，这要求教师的网络在上下文中与上下文中的学生模型平行进行微调。在目标环境中培训老师需要更多的时间和计算资源。在本文中，我们提出了不知道知识蒸馏的上下文，该蒸馏使用教师模型的知识而不在目标环境上进行微调。我们还提出了一种新的高效学生模型架构，用于知识蒸馏。提出的方法遵循两步过程。第一步涉及在不知道教师模型的不知道知识蒸馏的情况下预先培训学生模型。第二步涉及在图像检索的上下文上微调学生模型。为了显示拟议方法的功效，我们比较了检索结果。参数和否。在不同检索框架下，学生模型的运营与教师模型的运作，包括Deep Cauchy Hashing（DCH）和中央相似性量化（CSQ）。实验结果证实，所提出的方法在检索结果与效率之间提供了有希望的权衡。本文中使用的代码通过\ url {https://github.com/satoru2001/cukdfir}公开发布。

卷积神经网络（CNN）在许多计算机视觉应用中表现出非常吸引人的性能。通常使用基于随机梯度下降（SGD）优化技术进行CNN的训练。基于自适应动量的SGD优化器是最近的趋势。但是，现有的优化器无法在一阶时刻保持零平均值，并在优化方面挣扎。在本文中，我们提出了针对CNN的基于集中化的SGD优化器。具体而言，我们明确地将零均值约束强加于一阶力矩。提出的力矩集中化本质上是通用的，可以与任何现有的自适应动量优化器集成。提出的想法通过三种最先进的优化技术进行了测试，包括基准CIFAR10，CIFAR100和TINYIMAGENET数据集的ADAM，RADAM和ADABELIEF，用于图像分类。与建议的力矩集中化集成时，现有优化器的性能通常会提高。此外，提议的力矩集中化的结果也比现有的梯度集中化更好。使用玩具示例的分析分析表明，所提出的方法导致较短，更平滑的优化轨迹。源代码可在\ url {https://github.com/sumanthsadhu/mc-optimizer}中公开获得。